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trons2ref (Func_trans2ref) 
forward2tarqet_analyze (F unc_forward) 


Produces connected components of raw CELs. 
Vectorizes them into polylines. Then transforms 
vectorized polylines into reference aligned 
coordinated system (removing all non 
transformable data). Finally the window is 
forwarded into destination unresolved and the 
function that is attached to the window at its 
new destination is targeLanalyze. 
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_orea 
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vectorizes them into polylines. If in 
enable_color_masking mode is then connected 
components of raw color CELs are also 
computed, and then vectorize them into polylines. 
The function stp.reference of type 
: unc__stp_top_down„ref is called to learn bonding 
area and to create the single camera component 
of the reference including bonding pads windows, 
masks zones and stable features for registration, 
"hen all transformable data is transformed into 
reference aligned coordinate system (removing all 
non transformable data). Finally the window is 
brwarded into destination unresolved and the 
unction that is attached to the window at its 
new destination is bonding_analyze. 
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Func_angle_vectorizer<CEL>) 

circles_process 
(Func_circles_process) 

trans2ref (Funcjrans2ref) 

forword2balls_analyze 
(Func_forward) 


Produces connected components of raw CELs. 
Then vectorize them into polylines. This step is 
not really needed for the algorithm but it 
provides a nice picture to look at when 
visualizing the outcome of the algorithm. The 
function circles_process of type 
Func_circles_process is called to model the 
connected components as what are called 
generalized circles (balls). 
Then all transformable data is transformed into 
reference aligned coordinate system (removing all 
non transformable data). Finally the window is 
forwarded into destination unresolved and the 
function that is attached to the window at its 
new destination is balls_analyze. 
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the window is reported as defect. Then all 
transformable data is transformed into reference 
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forwarded. 


watchdog-func (Func.watchdog) 
trans2ref (Func_trans2ref) 

forward2reULin_camera 
(Func_forward) 




FIG. 8 



Function 
name 


Functions executed 


Remarks 


bolls analyze 


anolv7p rirrlp^ 
bafls_creote_ref 

balls_display (for display 
and debug only) 
merge_defects 

forward2balls_ref 
(Func_forward) 


Classifies circles hnspri nn da\n rnmlnn frnm 

three cameras. Creates circles reference. 
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Finally the window is forwarded into destination 
unified_balls_reference_defects_queue and the 
function that is attached to the window at its 
new destination is strip_balls. 


bonding_analyze 
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(Func_forword) 


Megre data from all three cameras into unified 
reference format. 

Finally the function is forwarded into destination 
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function that is attached to the window at its 
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polylines_comp_aaa_target 
(func_polylines_comp_aaa) 

defects_handler 
(Func_defects_handler) 

def ectsf ilter (Func.def ects.f ilter] 
forward2defects (Funcjorward) 


Function 

polylines_comp_aaa_target of type func_polylines_comp_aaa 

do a CEL2VEC comparison between reference polygons and 

online CELs based on application target tolerances. 

Function defects.handler of type Func.defectsjiandler is a 
post processing function that decides which of the defects 
reported by the various defect detectors (nick, protrusion, 
width defect CEL2VEC, excess/missing, etc.) are real 
Sip_defects. Defects found by a detector are rechecked 
according to zone specific application criteria. 

Function defectsfilter is of type Func.defeds Jilter is called 
to filter out any defects. It uses PIM.main and filters out all 
defects found in one or more regions of the following: 
unstable, mask.region, powerJine_default. 

Finally, if any Sip — defects are inside the window, then the 
window is forwarded into destination defects and the function 
that is attached to the window at its new dimension is nop 
(nothing to do). If no defects are found, then the window is 
not forwarded at all. 


pad_ref 


poly compare pads 


This function connected to this window type is similar to the 
function connected to the target window above. The only 
difference is that parameter applications are used for bonding 
pads to control the behavior of the CEL2VEC function. 


polylines - comp_aaa_target 
(func_polylines_comp_aaa) 

defects_handler 
[Func_defects_handler) 

defectsfilter(Func_defects_filter) 
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balls ins 
connect.open 

(Func_connected_components<CEL>] 

vectorize (is needed only for debugging one 
visualization of balls algorithms) 
Func_angle_vectorizer<CEL>) 

circles_process 
(Func__circles_process) 

trons2ref (Func_trans2ref) 

forward2balls_analyze 
(Func_forward) 


Produces connected components of raw CELs. 
Then they are vectorized into polylines. This step 
is not really needed for the algorithm but it 
provides a nice picture to look at when 
visualizing the outcome of the algorithm. The 
function circles_process of type 
Func_circles_process is colled to model the 
connected components and are called generalized 
circles (balls). Then all the transformable data is 
transformed into reference aligned coordinate 
system (removing all non transformable data). 
Finally the window is forwarded into destination 
unresolved and the function that is attached to 
the window at its new destination is 
balls_analyze. 


cavity 


watchdog 

watchdog_func (Func_watchdog) 
defectsfilter (Func_defects_filter) 
forword2defects (Func_forward) 


Function watchdog checks to see if there are 
CELs inside the window. Any CEL which is found 
inside the window is reported as a defect. 

Function defectsfilter is of type 
Func_defects_filter is called to filter out any 
defects. It uses PIM_main and filters out all 
defects found in one or more regions of the 
Following, unsiuoie, mo5K_region, 
power Jine_default. 

Finally, if any Sip_defects are found inside the 
window, then the window is forwarded into 
destination defects and the function that is 
attached to the window at its new destination is 
nop (nothing to do). If no defects are found, 
then the window is not forwarded at all. 
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poly compare and reject 

polylines_comp_and„reject 
(func_polylines__comp_aaa) 

Forward2defects (Funcjorward] 


Function polylines_comp_and reject does a 
CEL2VEC comparison between reference polygons 
and online CELs based on application target 
tolerances. This function has a very small limit 
to the number of excess CELs or missing 
envelopes permitted. If there is even a small 
change between reference target and online 
target overflow type defects are obtained which 
ore interpreted by the application as an indication 
that this frame should be rejected. 

Finally, if any Sip_defects is found inside the 
window, then the window is forwarded into 
destination defects and the function that is 
attached to the window at its new destination is 
nop (nothing to do). If no defects are found, 
then the window is not forwarded at all. 
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name 


Functions executed 


Remarks 


balls_analyze 


analyze_circles 

(Fnt\c circles nnnlv7p^ 

balls_compare2ref 
(Func_compare2ref) 

balls^ .display (for display 
and debug only. 

Func_display_balls_info) 

Merge.defects 
(Func_merge_defects) 

forward2defects 
(Func_forward) 


Classify circles based on data coming from three 
cameras. Compare unified circles to reference. 
Merge all defects from cameras into main unified 
data. 

Finally, if any Sip defects are found inside the 
window, then the function is forwarded into 
destination defects and the function that is 
attached to the window as its new destination is 
nop (nothing to do). If no defects are found, 
then the window is not forwarded at all. 



SIGNATURE (4 BYTES) 
HEADER (4 BYTES) 

data 

(cnt 4 bytes data item, value of cnt 
taken from header. Type of data items 
determined from "type" field of header) 



SIGNATURE (4 BYTES) 
HEADER (4 BYTES) 



data 



FIG. 18 




legal values for type 

0 : REPORT_NO_TYPE 

1 : REP0RT_C0L0R_DEFECT 

2 : REPORT_DEFECT 

3 : REPORT.CEL 

4 : REPORT_SNAP 


type (3 bits) 
of report element 






y (15 bits) 




cnt (12 bits) 




slice (2 bits) 




report 
element ^ 
In 



report 
element <! 
#(n+D 



FIG. 17 



FIG. 19 



CHANNEL 2 



CHANNEL_1 



TCP/IP 
CHANNEL 



610 



(DATA-SOURCEj) 







TASK_2 
(PRIORITY 1) 



c 



J: ^ 



600 



DMA 
CHANNEL 



3 



Cdata^sourcelO 







TASK_1 
(PRIORITY 1) 








DATA_S0URCE_3 



TASK_3 
(PRIORITY 2) 



(d^SOURCE^ 



I 



TCP/IP 
CHANNEL 



CHANNEL.3 



CHANNEL 2 



TCP/IP 
CHANNEL 







TASK.2 
(PRIORITY 1) 



1ST SIP 
PROCESS 



CHANNEL 1 



DMA 
CHANNEL 



C DATA_SOURCE_1 ) (DATA_SOURCE_2 j 







TASK_1 
(PRIORITY 1) 








DATA_SOURCE_3 




I 



TCP/IP 
CHANNEL 



CHANNEL.T 



FIG. 20 



c 



TCP/IP \ 
CHANNEL J 



Cdata_sourcel3^ 







TASK_3 
(PRIORITY 1) 







(DATA_S0URCE_4 



c 



I 



TCP/IP 
CHANNEL 



CHANNEL_3 
2ND SIP PROCESS 



FIG. 21A 






760 



FIG. 22A FIG. 22B FIG. 22C 



F 
















ii 








L 















L 



FIG. 22D FIG. 22E FIG. 22F 




L L 



FIG. 22G FIG. 22H 

SADDLE L 



FIG. 231 



o 1 



14 15 



a 

m 
y 
y 

m 
m 

~ 

a 
m 

C3 
Si 

o 

C3 




15 14 



1 0 



FIG. 24 



TANG_ 
WIDTH 830 



ENVELOPE 
BASE LINE 



WIDTH 820 




ENVELOPE 
800 



TANG_ 
WIDTH 830 



FIG. 25 




860 




Oi 0 



o, a 



§3 



0i a 



0| 0 



z: 



36 



a, a 



0| o 



o, o 



0,0 



Di o 



o. o 



,870 

FIG 



26 



FIG. 27 




882 



WIDTH 
MEASUREMENT 



900- 



FIG. 28A 



920 



WIDTH 
MEASUREMENT 



SKELETON 



910 




920 

SKELETON 
V 910 

FIG. 28B 




FIG. 31 



IDENTIFY CELS 


IN AN ENVELOPE 


1100 








DENOTE CELS HAVING SAME 
GENERAL DIRECTION AS A POLYLINE ' 
AS BEING "MATCHING" CELS 


^1200 








MICROREGISTER REFERENCE FEATURE 
AND ONLINE FEATURE IN INSPECTION " 
WINDOW TO MINIMIZE LEAST SQUARE 
DISTANCES BETWEEN MATCHING CELS 
AND LINE VECTOR 


^1300 



FIG. 33 



USER SPECIFIES CRITERIA DEFINING 
REGULAR AND MINOR DEFECTS 



3305 



USER SPECIFIES MAXIMUM ACCEPTABLE 
NUMBER (X) OF EACH TYPE OF MINOR DEFECT 



3315 



3310 



INSPECT IMAGE 



3320 




NO 



r 



3335 



3330 



YES 



DEFECT REPORT 




r 



3325 



NO 



IMAGE 
OK 



r 



3340 



IDENTIFY NUMBER (N) OF EACH 
TYPE OF MINOR DEFECT 



3345 



YES 




NO 



oo 

K") 




cr> 
6 




FIG. 40 



IE 



expert 



File Operote 



Current_camera_olignment_transform.O 10.967886 0,0001 II II 0 
Current_comera_alignment_transform_1 10.970368 -0-QQQQ43 0 
Current_comero„oliqnment Jransform_2 10.976427 -0.000074 0 



Current_camera_overlaps 1|0 147.745( {147.368 314.107 
CurrenLcamera.sections IjO 2022j j72 1938j j156 2096 ■ 



Learn_camero_alignmenLtransform_0 |1.004721 -0.000014 -0. 



Learn_camera_alignmentJransformJ 1 1.001 51 8 0.000049 -0.0 
LeQrn,comera_olignmen t_tronsfQrm_2 1 1.006841 -0.00098 -0. 
Learn_camera_overlaps 1{0 271.163j {272.03 273.346j |271 
Leorn,comero_se ctions 1{0 1960j {135.596 1959j {135 
RepeaLAdjusLX I 0.0 
Repeat_Adjust_Y I 0.0 
RepeaLLength 1 1 00 
RepeaLRepeaLX j 1 
RepeaLRepeaLY H 



RepeaLSize.X [100 
RepeoLSize. Y |T00~ 
Repeat.Width 0T 



RepeaLX.offset |0.0 
Repeat_Y_offset fO0" 
balls_adlust_to_nominal [yes 



balls_circle_fiLsensitivity |40.0 
balls_moximum_alignment_shift |60.0 



bolls_moximum_registrg{io n_shift |60.0 
bolls_percentoge_circle_fit |60.0 



camera_pixels_per_mm [66.667 
camera.pixels^sizejn^micron [T5~ 
camera_width |20 96 
ceLsubpixeLsize |16.0 
chonnei_rep_dump 



no 



chonnel_red_io_type | File 
channel_rep_port_number 
channel_sdd_dump 



6001 



no 



Apply | Comrodd Config | 



FIG. 41 



expert 



n 



File Operate 



eyence 



^ 



channel_sdd_dump I no 
chonneLsddJoJype I FILE 
channel_sdd_port_number (6002 
defect_unifying_distance_mils 



8 



defecLwindows Jile | /icp/fromes/defecLwindows 
dir_of_defects |/tem/panel 
dir_of_ref_panel | /tmp/ponel 



WARNING 



dir_oLsip_con fig | /project/icp/dvlp/linux/lib/sip/config" 
inpuLrepJile I /icp/tmp/ponei/dump.cel.intel 
input_snap_file | /icp/tmp/panel/dump.snp.intel 
iog_file_of_sip [ std err 
iogJeveLchannels 
log_level_functions 
logJeveLgener al [WARNING 
log.leveLtosks [WARNING 
max_registration_toleronce 
multLprocess 
output_rep_file 
outpuLsddJile 



WARNING 



45.0 



no 

/tmp/ponel/ dump 



/tmp/panel/output.sdd 



reg DlSlANCES IN PIXELS 



OF 
ottme 



reg_boot_time_limit 
reg_dynomic_model 
reg_dynamic_optimaLdelto_y 1 500 
reg_dynamic_optimaLpoints 
reg_feotures_noise 



20F 



reg_matcherjable | matcher- table.bga 
reg_max_paneLrotation 
reg_max_poneLshift 



1.0 



reg_max_points_for_b oot |200 
reg_max_y_for_boot [150 0 
reg_min_points_for_boo t 1 1 50 
reg_required_accuracy f075 
reg_unifying_distance [E 



Apply | 



Comradd Confi 



FIG. 42 



File Operote 



expert 



/SipVo^ 



logjeveljunctions | WARNING 



log_level_generol [WARNING 



log Jeveljasks (WARNING 
max_registration_tolerance 
multLprocess 
outpuLrepJile 
output_sdd_file 



mo 



no 



/tmp/panel/dump" 



/tmp/panei/output.sdd" 



reg|REG DISTANCES I N PIXEL? 
reg_booLtime_limit 1 0.5 



reg_dynamic_model | attme ' 
reg_dynamic_optimaLdelto_y 1 500 
reg„dynomic_optim aLpoints |2Q0 
reg_features_noise f? 



reg_matcher_table | matcher-toble.bgo 
reg_mox_poneLrototi on 1 1 .0 
reg_max_paneLshift | 200 
reg_max_points_for_b oot [200 
reg_mox_y_for_boot [150 0 
reg_min_points Jor„boo t 1 1 50 
reg_required_accuracy 1 0.5 
reg_unifying_distance [B 



show_alLem_defects (false 



show„alLhw_defects (false 



show_all_lw_defects I false 



sip_buffer_size (10000000 



sip_config_dir | /ho me/mQlcolm/lcp/Dvlp/alfi/lib/sip/config" 
sip_dma_buffer„size (1 0000000 
sip_maxjine_in_scan (14664 
sip_mox_number_of_defect_report 
type_enable_color_masking [yts 
type_monuoLpods_l earninq [no 



1000 



type_panel_polarity fl 
type_power_lines_area_threshold 



50O0 



Apply 



Comradd Config 




FIG. 43 



ASYNCH 
CONSUMER 




CONSUMER (RED) 
DATA 

source 



PROCESSING 
MODULE 



XL 



DATA 
SOURCE] ^ 

PRODUCER (WRITE) 



FIG. 44 



i''SHARED N ! 
i DATA 
^SOURCE, 




j SHARED i 
i DATA 
^SOURCE 



PRE-EMPTIVE 
PROCESSING 
MODULE 



f'SHARED^i 
i DATA 

Lsourcl 



DATA 
SOURCE 




DATA 
SOURCE 



COOPERATIVE 
PROCESSING 
MODULE 



DATA 
SOURCE 



FIG. 45 



PROCESSING 
MODULE 



COMPOSITE 
PROCESSING 
MODULE 



INPUT 
CHANNEL 



OUTPUT 
CHANNEL 



TASK TRANS- 
FORMATION 
GENERATOR 




FIG. 46 



# 



FIG. 47 



DMA CHANNEL 



'IStrearrfi ! ! 
DATA j ! j 



'.SOI 

< 


RCE>- 
) 


PRE-EMPTIVE 
PROCESSING 
MODULE 


i'SHAF 


!EDM! 



DATA 
'.SOURCES 



FILE CHANNEL 



IStream 
DATA 



SOU 



RCEX 



COOPERATIVE 
PROCESSING 
MODULE 



DATA 
SOURCE 



FIG. 48 



TCP/IP CHANNEL 



r SHARED M! 



DATA 



! 1 i 



'.SOURCES 



PRE-EMPTIVE 
PROCESSING 
MODULE 

L , J 



'OStreairii 
DATA 



•! 1 1 
j!j 

sSOURCE,^ 



FILE CHANNEL 



DATA 
SOURCE 







COOPERATIVE 
PROCESSING 
MODULE 







DATA 
ISOURCEX 




* 



FIG. 49 



o 

*o 

Co 
hi 

y 

M 

m 
m 

E 

£3 
CO 
C3 
N 
C3 
C3 



DATA 
SOURCE 



REGISTRATION 
PROCESSING 
MODULE 



.''TRANSFORM"^ 
S DATA 
L SOURCE 



i EXCESS-MISSING ) 
DATA 
SOURCE 



FIG. 50 



INTERFACE 
TO THE 
APPLICATION 
MODULE 



SWITCHBOARD 
LOOP 



PROCESSING 
MODULE 
MANAGER 



r~rr 



< DATA 



SOURCE 




( DATA )]' 

1 source !>' 



J DATA iji 
J SOURCE J \) 



A 



PRE-EMPTIVE 
PROCESSING 
MODULE 



_.._J 

PRE-EMPTIVE 
PROCESSING 
MODULE 



r ___JL____, 

PRE-EMPTIVE 
PROCESSING 
MODULE 




FIG. 51 



IStream 

DATA 
SOURCE 



i CHANNEL 
, PROCESSING 




{ "shared" 1 1 \ ( sharedT^ ' 



, , SHARED ) j i f SHARED 1 J i ( SHARED 1 ] i { SHARED \ 1 
J DATA Ji i J DATA j, j DATA ji'j DATA j ! j j DATA ■!'• DATA 'I' 
{ SOURCE ]} [ SOURCE j>' \ SOURCE j>' SOURCE^ SOURCE \y \ SOURCE j>' 





f SHARED^ 
j DATA ',, 
{ SOURCE j>' 



.!> 



- — . ) 
[ SHARED) i 
J DATA Ji! 
[ SOURCE^ 




[ SHARED! : i 

i ! i 



DATA 



> J 



[ SOURCE j> 




PRE-EMPTIVE 
PROCESSING 
MODULE 

"""IT"" 

( OStream) 
! DATA ! 
I SOURCE J 



C3 

m 
y 
u 

in 
m 

s 

C3 
CO 

£3 

Si 

C3 
Cj 



FIG. 52 

SIP_ROOT 



GENERAL 
(MULTICLIENT SUPPRT) 



OO-SERVER 
(MULTICLIENT SUPPRT) 




INCLUDE 

HARDWARES AND 
OTHER BASIC INC. 





FIG. 53 



RSERROR 



CLEAR 



UINITS 




INITS 



SIP_END_0F> 
SCAN_0K 




SIPCTOR 



CLOSE 




OPEN 



SIP_C0MM_0PEN) 



CLEAR 




SIP.LOADED 



LOAD 



UINITS 





SIP_READY_ 
TO.SCAN 



INITS 



SCAN 



SCAN ENDED SUCCESSFULLY 



SCAN ENDED WITH AN ERROR 



sip.endIofN 

SCAN_ERRO^y 




STOP 



SCAN.NB 




SIP_SCANNING 




FIG. 54 



• DENOTES THE COLOUR DEFECT 
REPORT DUE TO WHICH THE 
SNAP OF SIZE 48X48 IS RECORDED. 



FIG. 55 



. DENOTES A SIGNAL COLOR_DEFECT REPORT 
I | DENOTES AREA OF THE SNAP. 



X s rgb 



FIG. 56 

REPRESENTS A SINGLE 32-BIT SNAP REPORT WORD. WHICH 
CONTAINS THE 8-BIT X-COORDINATE (X s ) AND THE VALUES 
OF THE RED (r) GREEN (g) AND BLUE (b) 



START OF 
A NEW LINE 







N=l 








N=2 








N=3 








0 


• • • 


13 


14 


15 


0 


• ■ • 


14 


15 


0 


m m • 


15 




0 


Orgb 


* • • 


0 rgb 


Orgb 


Orgb 


1 rgb 


• * « 


1 rgb 


1 rgb 


2 rgb 


• • * 


2 rgb 


Orgb 




• # • 


x=13 


x=14 


x=15 


x=16 


* * * 


x=30 


x=31 


x=32 


• • « 


x=47 





START OF 
A NEW LINE 



METAL 
COVERED BY 
SOLDER MASK 




METAL REGIONS 



METAL REGIONS 



FIG. 57 



METAL 
COVERED BY 
SOLDER MASK 



FIG. 58 REGION TO BE IGNORED 




o 

m 
U 
y 
\i 

m 

en 

s 

o 

CO 

Q 

M 

£3 
C3 



FIG. 60A 




LIST OF REGIONS 








PIM 

MAP GENERATOR 








UNIFIED REGIONS' 
MAP 





FIG. 59 




FIG. 60B 




ON-LINE EVENTS ) 



('UNIFIED REGION^ 







POINT+TYPE 






FILTER 


PIM 

LOCATION QUERY 




YES/NO 










FILTERED EVENTS; 



FIG. 61 




Y VECTOR 
LONG Y 



FIG. 62 




WHERE THE GRAPHIC INTERPRETATION IS: 
id O=0 Wi 



id 



n 



: 0 x l 



n 



FIG. 63 



PIM SIP 



PIM 



ill 



QUERY TABLES 



QT1 




QT2 




QT3 



SHAPE FILE READER 






SHAPE 







SHAPE FILE READER 




DPOLYLINE 




DRECT 


DCIRCLE 



FIG. 64 



COLOR-DEFECTS 



SNAPS 



TASK_TRIGGER_HANDLER 



FIG. 65 

— FOR (Y=Y_FIRST;Y<=Y_LAST;++Y) 

CLUSTERING: 1) CLUSTER CURRENT LINE 
2) CHECK AND CLOSE 



IF Y=Y_LAST: 






CUT THE APROPREATE SNAP REGIONS 










UPDATE WINQUEUE (PACKING) 










10 



FIG. 66 



15 



20 



25 



30 































































































c 


:lus 


ITER 


2 








































IP 
mm 






















n 


I ITQ 
LUo 


TFP 














































CLl 


JSTI 


CR2 






















































































■I 
























CL 


UST 


ER2 










































■ 



















































































































10 



- PIXEL WITH COLOR DEFECT REPORT Q - BOUNDING RECTANGLE 



THE RUN LENGTH STRUCTURE FOR THE CASE WHERE THE SNAP REGION 
IS THE BOUNDING RECTANGLE ENLARGED BY TWO PIXELS ON EACH SIDE: 
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